2013-03-29T00:13:27 *** dvladim has quit IRC (Ping timeout: 260 seconds) 2013-03-29T00:23:46 *** coeus_ has quit IRC (Ping timeout: 256 seconds) 2013-03-29T00:24:09 *** coeus_ has joined #aichallenge 2013-03-29T01:37:07 *** yoden has quit IRC (Read error: Connection reset by peer) 2013-03-29T01:37:37 *** yoden has joined #aichallenge 2013-03-29T01:39:27 *** Uvs has joined #aichallenge 2013-03-29T02:25:18 *** mceier has quit IRC (Quit: leaving) 2013-03-29T02:32:31 *** janzert has quit IRC (Read error: Connection reset by peer) 2013-03-29T02:34:10 *** janzert has joined #aichallenge 2013-03-29T02:37:06 *** UncleVasya has joined #aichallenge 2013-03-29T02:40:09 *** Uvs has quit IRC (Ping timeout: 258 seconds) 2013-03-29T02:48:01 *** UncleVasya is now known as Uvs 2013-03-29T02:50:38 *** Accoun has quit IRC (Ping timeout: 252 seconds) 2013-03-29T02:54:40 *** Accoun has joined #aichallenge 2013-03-29T03:16:24 *** mceier has joined #aichallenge 2013-03-29T03:48:47 *** epicmonkey has joined #aichallenge 2013-03-29T04:12:09 *** epicmonkey has quit IRC (Ping timeout: 258 seconds) 2013-03-29T04:49:44 *** sigh has joined #aichallenge 2013-03-29T05:07:20 *** thestinger has quit IRC (Quit: WeeChat 0.4.0) 2013-03-29T05:13:49 *** epicmonkey has joined #aichallenge 2013-03-29T05:26:54 *** mviel has joined #aichallenge 2013-03-29T05:28:21 *** warrens has joined #aichallenge 2013-03-29T05:37:15 *** warrens has quit IRC (Ping timeout: 258 seconds) 2013-03-29T05:42:57 *** Accoun has quit IRC (Remote host closed the connection) 2013-03-29T05:52:12 *** Accoun has joined #aichallenge 2013-03-29T06:05:21 *** HornyReaper has joined #aichallenge 2013-03-29T06:36:26 *** mcstar has joined #aichallenge 2013-03-29T06:37:17 *** Uvs has quit IRC (Ping timeout: 246 seconds) 2013-03-29T06:43:11 *** kilae has joined #aichallenge 2013-03-29T06:47:00 *** loglog has quit IRC (Ping timeout: 264 seconds) 2013-03-29T06:58:24 *** coeus_ has quit IRC (Ping timeout: 264 seconds) 2013-03-29T06:58:24 *** bqf has quit IRC (Ping timeout: 264 seconds) 2013-03-29T06:59:00 *** bqf has joined #aichallenge 2013-03-29T07:03:03 *** coeus_ has joined #aichallenge 2013-03-29T07:24:28 *** cyphase has quit IRC (Quit: http://www.cyphase.com/) 2013-03-29T11:09:28 *** mceier has quit IRC (Quit: leaving) 2013-03-29T11:15:19 *** Uvs has joined #aichallenge 2013-03-29T11:29:13 *** epicmonkey has quit IRC (Ping timeout: 240 seconds) 2013-03-29T11:31:44 *** sigh has quit IRC (Remote host closed the connection) 2013-03-29T11:58:41 *** mceier has joined #aichallenge 2013-03-29T12:05:50 *** loglog has joined #aichallenge 2013-03-29T13:08:35 *** rofer has quit IRC (Ping timeout: 260 seconds) 2013-03-29T13:09:10 *** mviel has quit IRC (Ping timeout: 258 seconds) 2013-03-29T13:22:53 *** mviel has joined #aichallenge 2013-03-29T13:24:57 *** thestinger has joined #aichallenge 2013-03-29T13:25:47 *** rofer has joined #aichallenge 2013-03-29T13:54:54 *** cyphase has joined #aichallenge 2013-03-29T13:59:33 *** smiley1983 has joined #aichallenge 2013-03-29T13:59:37 'pile of shit' is the phrase im looking for 2013-03-29T14:01:42 \o 2013-03-29T14:02:16 o/ 2013-03-29T14:43:13 *** mviel has quit IRC (Remote host closed the connection) 2013-03-29T15:08:06 thestinger: i was referring to this http://sprunge.us/iDXY?python 2013-03-29T15:08:24 it is getting so ugly... 2013-03-29T15:08:39 you made the mistake of writing a python program longer than 100 lines 2013-03-29T15:08:49 barely, but yeah 2013-03-29T15:08:59 im not sure if python is to blame 2013-03-29T15:09:31 i just dont like the fact that i have to use too much kind of technology 2013-03-29T15:09:32 mcstar: can you use multiprocessing.Manager? 2013-03-29T15:09:40 not sure 2013-03-29T15:09:43 what does it do? 2013-03-29T15:10:00 makes a socket in /tmp and lets you send pickled objects 2013-03-29T15:10:04 abtracts it all 2013-03-29T15:10:07 abstracts* 2013-03-29T15:10:47 thestinger: there is Pipe and Queue in multiprocessing 2013-03-29T15:10:55 http://docs.python.org/3.3/library/multiprocessing.html#managers this is higher level though 2013-03-29T15:11:30 idk, that part is good enough 2013-03-29T15:11:39 * thestinger shrugs ;p 2013-03-29T15:11:42 im not satisfied witht he bluelet's part 2013-03-29T15:12:10 and it is hard to see what code runs where 2013-03-29T15:12:36 there is one instance of HQ that is in the main thread, this is user interactive 2013-03-29T15:12:47 there is an instance of Manager that is in a background thread 2013-03-29T15:12:56 and then there are functions that operate on remote machines 2013-03-29T15:15:23 thestinger: have you actually tried some kind of concurrent programming in python? 2013-03-29T15:15:42 mcstar: multiprocessing.Manager and horrible async stuff 2013-03-29T15:15:47 (something other, than a web framework, that usually abstracts it away) 2013-03-29T15:15:55 horrible async callbacks. 2013-03-29T15:15:58 twisted. 2013-03-29T15:16:13 mcstar: you can use Manager for remote stuff too 2013-03-29T15:16:21 i cant 2013-03-29T15:16:25 and it's much simpler than threading, so shared mutable state 2013-03-29T15:16:27 the remote side is given 2013-03-29T15:16:27 mcstar: why? :P 2013-03-29T15:16:30 oh 2013-03-29T15:16:44 no shared mutable state* 2013-03-29T15:16:45 i only talk to a local controller 2013-03-29T15:16:49 ah 2013-03-29T16:03:27 *** thestinger has quit IRC (Quit: WeeChat 0.4.0) 2013-03-29T16:05:58 *** smiley1983 has quit IRC (Ping timeout: 245 seconds) 2013-03-29T16:40:14 *** epicmonkey has joined #aichallenge 2013-03-29T16:55:22 *** thestinger has joined #aichallenge 2013-03-29T17:47:33 *** epicmonkey has quit IRC (Ping timeout: 240 seconds) 2013-03-29T18:09:46 *** Uvs has quit IRC (Read error: Connection reset by peer) 2013-03-29T18:10:06 *** kilae has quit IRC (Quit: ChatZilla 0.9.90 [Firefox 19.0.2/20130307023931]) 2013-03-29T18:55:08 *** mcstar has quit IRC (Quit: mcstar) 2013-03-29T19:16:05 *** sigh has joined #aichallenge 2013-03-29T19:34:59 *** HornyReaper has quit IRC (Remote host closed the connection) 2013-03-29T21:00:44 *** chris__0076 has joined #aichallenge 2013-03-29T21:04:18 *** Chris_0076 has quit IRC (Ping timeout: 245 seconds) 2013-03-29T21:26:35 *** mcstar has joined #aichallenge 2013-03-29T21:26:46 thestinger: i have a python question 2013-03-29T21:26:52 surprise 2013-03-29T21:26:53 okay 2013-03-29T21:27:11 lets say i have an object, anything 2013-03-29T21:27:20 basically everything is a reference 2013-03-29T21:27:40 except things that aren't ;p 2013-03-29T21:27:40 so if i have the number(memory location) of that object i can do whatever i want with it 2013-03-29T21:28:02 I guess? :P 2013-03-29T21:28:05 most things are on the heap, so there are pointers to them 2013-03-29T21:28:12 anyway 2013-03-29T21:28:36 what i want, is send over the multiprocessing pipe or queue a reference of something 2013-03-29T21:28:44 the damn things tries to pickle it 2013-03-29T21:28:55 and it fails, since im sending functions or class methods 2013-03-29T21:29:21 but if i could just send over the pointer to that function, which is an int, that could work 2013-03-29T21:29:41 unfortunately i havent figured out a way to actually do it 2013-03-29T21:30:00 ctypes? 2013-03-29T21:30:05 ctypes.addressof 2013-03-29T21:30:10 lol 2013-03-29T21:30:23 mcstar: you know this is a horrible idea right ;p 2013-03-29T21:30:30 why? 2013-03-29T21:30:41 why would it be? 2013-03-29T21:30:49 it makes things easier 2013-03-29T21:31:04 i dont have to introduce a lot of enum like values 2013-03-29T21:31:50 instead of setting up a value, that maps to a function, i just send the address of the function 2013-03-29T21:32:06 you can create types with type() :P 2013-03-29T21:32:53 mcstar: what exactly do you want to do though 2013-03-29T21:33:00 just what i wrote 2013-03-29T21:33:11 what can you do with the pointer to the function though? 2013-03-29T21:33:19 I don't think you can call it 2013-03-29T21:33:25 why couldnt i? 2013-03-29T21:33:31 thats exactly what i want to do 2013-03-29T21:33:48 how would you call it? :P 2013-03-29T21:34:00 thestinger: thats what im asking you! 2013-03-29T21:34:08 how could i call it? 2013-03-29T21:34:26 the damn __repr__ even has the memory address of the function 2013-03-29T21:34:53 let me describe what i want 2013-03-29T21:35:05 thread 0: controls thread 1 through a pipe 2013-03-29T21:35:12 it can send commands 2013-03-29T21:35:25 like Exit, DoThis or DoThat 2013-03-29T21:35:48 now in that object that is instantiated in thread 2 i have to map these things 2013-03-29T21:35:55 I guess you could make a ctypes function pointer, and call that? 2013-03-29T21:35:55 Exit -> self.exit 2013-03-29T21:35:58 ... 2013-03-29T21:36:00 http://docs.python.org/3.4/library/ctypes.html#callback-functions 2013-03-29T21:36:31 i wonder if i could just disable pickling 2013-03-29T21:36:55 or use threads 2013-03-29T21:37:08 but python threads aren't very parallel... 2013-03-29T21:37:20 what do you mean? 2013-03-29T21:37:23 concurrent, but not very parallel :) 2013-03-29T21:37:27 they are as parallel as the CPU 2013-03-29T21:37:32 mcstar: no 2013-03-29T21:37:35 yes 2013-03-29T21:37:37 they context-switch but they rarely run at the same time 2013-03-29T21:37:39 just dont lock 2013-03-29T21:37:45 they do lock 2013-03-29T21:37:50 because of the GIL 2013-03-29T21:38:11 it won't lock if it's in a C call that specifically doesn't ask for a lock 2013-03-29T21:38:13 why would threads lock? 2013-03-29T21:38:21 if theres no communication theres no locking 2013-03-29T21:38:29 because cpython only knows how to run one thread of code at a time 2013-03-29T21:38:40 http://wiki.python.org/moin/GlobalInterpreterLock 2013-03-29T21:38:48 whats the difference with multiprocessing? 2013-03-29T21:38:58 i thought it was just a better interface over threads 2013-03-29T21:39:01 processes are each a whole python interpreter 2013-03-29T21:39:08 so they are truly parallel 2013-03-29T21:39:23 mcstar: it's essentially the same, except you can't share memory with processes 2013-03-29T21:39:28 ok 2013-03-29T21:39:32 message passing is regarded as cleaner 2013-03-29T21:39:38 well, thats not true 2013-03-29T21:39:44 but it has to pickle everything 2013-03-29T21:39:48 well 2013-03-29T21:39:49 message passing within a process is great 2013-03-29T21:40:02 messages passing between processes..... well, serializing everything 2013-03-29T21:40:11 i just checked the address of the code and it was the same in both threads 2013-03-29T21:40:13 so they share that 2013-03-29T21:40:22 mcstar: because when you fork(), it does copy-on-write 2013-03-29T21:40:35 but the granularity is something like pages 2013-03-29T21:40:57 mcstar: if you use multithreading you don't need anything pickles 2013-03-29T21:41:00 pickled* 2013-03-29T21:41:05 you just pass objects and functions around 2013-03-29T21:41:15 it will be fine for I/O (it's concurrent) but not for using all the cores 2013-03-29T21:41:21 what? 2013-03-29T21:41:25 i dont get it 2013-03-29T21:41:34 with multithreading everything has to be pickled 2013-03-29T21:41:37 no 2013-03-29T21:41:51 with multiprocessing everything has to be bytes, so it pickles stuff 2013-03-29T21:42:10 multithreading -> can pass anything around, can share memory, but not parallel because of the GIL 2013-03-29T21:42:33 multiprocessing -> only messages passing, and only bytes (so it needs to serialize) 2013-03-29T21:42:34 sry that was my mistake, i meant multiprocesssing 2013-03-29T21:42:56 yes, i agree 2013-03-29T21:43:20 but let me rewind a bit 2013-03-29T21:43:26 threads > processes usually, but you need to be disciplined about data races 2013-03-29T21:43:33 and threads in python suck 2013-03-29T21:43:34 ;[ 2013-03-29T21:43:51 thestinger: why does the same class have the same address in both processes? 2013-03-29T21:44:04 when you fork a child process, it shares 100% of the memory 2013-03-29T21:44:08 until it starts touching stuff 2013-03-29T21:44:11 then it copies 2013-03-29T21:44:17 but that will never change 2013-03-29T21:44:25 it will if a write near the class happens 2013-03-29T21:44:27 in memory 2013-03-29T21:44:32 you can't rely on it afaik 2013-03-29T21:44:35 hm 2013-03-29T21:44:37 it probably will still the same 2013-03-29T21:44:41 but it might not 2013-03-29T21:44:45 i see 2013-03-29T21:45:02 i didnt consider that memory references are not persistent in python 2013-03-29T21:45:28 can something like this happen if i dont use any multiprocessing? 2013-03-29T21:45:35 no 2013-03-29T21:45:39 threads share the same address space 2013-03-29T21:45:46 no i dont mean that 2013-03-29T21:45:51 in single threaded code 2013-03-29T21:45:54 single process 2013-03-29T21:45:58 possibly 2013-03-29T21:46:05 I really don't think you can rely on addresses 2013-03-29T21:46:16 too bad 2013-03-29T21:46:21 also a good gc moves stuff around 2013-03-29T21:46:23 python lacks that ;p 2013-03-29T21:46:35 yeah, the gc 2013-03-29T21:46:52 hm 2013-03-29T21:47:08 now this is interesting, and i havent considered this before 2013-03-29T21:47:30 a GC can move around a tree or something heavily branched 2013-03-29T21:47:52 but in doing so what does it do? 2013-03-29T21:48:08 add/subtract the same amount to/from every pointer 2013-03-29T21:48:27 or can change them individially 2013-03-29T21:48:41 mcstar: I don't think cpython's does that ever though 2013-03-29T21:48:48 it's really naive 2013-03-29T21:49:00 it just collects cycles, refcounting does the rest 2013-03-29T21:49:00 lets leave python for a sec 2013-03-29T21:49:04 a gc can move everything around if it manages all the pointers 2013-03-29T21:49:13 and it does that in practice? 2013-03-29T21:49:16 yes 2013-03-29T21:49:17 like java 2013-03-29T21:49:19 jvm 2013-03-29T21:49:20 yeah 2013-03-29T21:49:40 it does it to compact the heap 2013-03-29T21:49:43 reduces fragmentation 2013-03-29T21:49:50 yeah i read all these 2013-03-29T21:50:02 but it didnt occur to me that it has to change the pointers... 2013-03-29T21:50:11 iis quite silly 2013-03-29T21:50:16 premature enter 2013-03-29T21:50:18 some languages can't have a moving gc 2013-03-29T21:50:21 i wanted to say: 2013-03-29T21:50:24 if they have unmanaged ptrs 2013-03-29T21:50:27 i might seem quite silly... 2013-03-29T21:51:16 mcstar: gc is really scary ;p 2013-03-29T21:51:19 it has to flush registers too 2013-03-29T21:51:23 a pointer might be hiding in there 2013-03-29T21:51:27 thestinger: can the os do something similar? 2013-03-29T21:51:27 it has to walk the stack 2013-03-29T21:51:30 i guess not 2013-03-29T21:51:33 mcstar: yes 2013-03-29T21:51:37 processes use virtual memory 2013-03-29T21:51:37 oooh 2013-03-29T21:51:44 technically, they don't know real addresses 2013-03-29T21:51:48 yeah, but it has to move stuff contiguously 2013-03-29T21:53:29 but when in asm i write addresses, is that always wrt a base address? (which i see as 0) ? 2013-03-29T21:54:21 i guess it wouldnt be too secure if programs were aware of real memory addresses 2013-03-29T21:54:39 mcstar: I'm not sure how directly VM maps to real addresses 2013-03-29T21:54:45 but consider stuff like swap 2013-03-29T21:54:49 but then this all requires another dereferencing for every access 2013-03-29T21:55:04 can have multiple swap files and stuff is mapped in and out of real memory 2013-03-29T21:55:07 so it can't be very direct 2013-03-29T21:55:50 thestinger: and consider that the mapping must be done by the OS, but the program has to run without the intervention of the OS for some time, until a timed interrupt occurs, so how does it translate addresses to real memory? 2013-03-29T21:56:06 * thestinger shrugs ;p 2013-03-29T21:56:21 mcstar: https://en.wikipedia.org/wiki/Memory_management_unit 2013-03-29T21:56:46 mcstar: and memory has permissions 2013-03-29T21:56:57 rwx basically... 2013-03-29T21:57:05 so this translation happens in hardware 2013-03-29T21:57:14 yeah 2013-03-29T21:57:17 a lot, at least 2013-03-29T21:57:28 not any kind of expert on this stuff ;p 2013-03-29T21:57:33 * thestinger just has a loose understanding 2013-03-29T21:57:54 i have a looser one 2013-03-29T21:59:58 *** sigh has quit IRC (Remote host closed the connection) 2013-03-29T22:03:38 mcstar: http://ix.io/4WE example 2013-03-29T22:03:47 address of that variable is the same after a fork 2013-03-29T22:03:51 the child mutates it 2013-03-29T22:03:57 when the child exits, the parent wakes up 2013-03-29T22:04:05 it reads the value, and it *wasn't* mutated 2013-03-29T22:04:13 the child mutating it does a copy-on-write copy for the child 2013-03-29T22:04:18 it only pretends the address is the same 2013-03-29T22:04:46 hm 2013-03-29T22:05:04 so i have to do the translation 2013-03-29T22:05:11 probably ;p 2013-03-29T22:05:13 thestinger: well, thanks 2013-03-29T22:05:39 I think it's fine to *read* anything 2013-03-29T22:05:48 but I don't really know the details about python's inner workings 2013-03-29T22:05:53 like sci-fi? 2013-03-29T22:06:01 something like that ;p 2013-03-29T22:06:14 thestinger: you know quite much 2013-03-29T22:06:17 mcstar: anyway it's all scary 2013-03-29T22:06:29 for example, fork() gives you a view of the whole old address space 2013-03-29T22:06:39 but it only actually copies over a single thread 2013-03-29T22:06:46 so it can really mess up locking. 2013-03-29T22:07:06 5 threads running and one forks -> all the locks the 4 other threads locked are still locked, and won't ever be unlocked 2013-03-29T22:07:11 python probably hides that 2013-03-29T22:07:13 C doesn't ;p 2013-03-29T22:07:24 anyway concurrency scares me 2013-03-29T22:07:38 unless the language is built around it 2013-03-29T22:07:46 <3 STM 2013-03-29T22:08:17 which is threads, with safety layered on 2013-03-29T22:08:18 (yeah, thats quite an exaggeration, i only just wrote anything STM related) 2013-03-29T22:08:45 mcstar: the C++11 stuff is basically STM 2013-03-29T22:08:45 well, not really, they can be OS processes too 2013-03-29T22:08:55 ghc runtime can map threads to real processes 2013-03-29T22:09:00 mcstar: processes can't write to each other's memory tho, it fakes it all 2013-03-29T22:09:08 it's doing message passing under the hood 2013-03-29T22:09:12 that doesnt matter from the user's perspective 2013-03-29T22:09:35 thestinger: really? STM in c++? 2013-03-29T22:09:43 mcstar: yes. 2013-03-29T22:09:51 thats weird 2013-03-29T22:10:00 mcstar: http://en.cppreference.com/w/cpp/atomic 2013-03-29T22:10:02 how is that implemented? 2013-03-29T22:10:08 much lower level 2013-03-29T22:10:12 does it have retry? 2013-03-29T22:10:12 mcstar: well CPUs have atomic instructions 2013-03-29T22:10:20 but it will fall back to a lock if it has to 2013-03-29T22:10:24 that has nothing to do with STM 2013-03-29T22:10:49 why not? :P 2013-03-29T22:10:52 STM is *software* TM, it can keep a log, and roll back 2013-03-29T22:11:18 it is not like an atomic operation, that openmp offers too 2013-03-29T22:11:35 (openmp offers real atomic cpu increment/decrement) 2013-03-29T22:11:38 mcstar: it's STM just a transaction that's only visible until it's complete? 2013-03-29T22:11:44 I didn't think it *had* to be possible to roll back 2013-03-29T22:11:47 some stuff can't fail 2013-03-29T22:11:53 isn't STM* 2013-03-29T22:12:07 STM guarantees that the worls is always consistent 2013-03-29T22:12:27 sometimes it keeps the promise by rolling back previous operations 2013-03-29T22:12:38 so you cant have IO and STM at the same time 2013-03-29T22:13:05 you have to call `atomically` from IO, that does an STM operation 2013-03-29T22:13:18 ofc, that can be a whole chain of them 2013-03-29T22:13:24 but they cant have IO inside 2013-03-29T22:13:25 it just seems like a higher level version of atomics to me ;p 2013-03-29T22:13:35 composable atomics 2013-03-29T22:13:45 (ofc there is an unsafeIOtoSTM function that we never ever use_ 2013-03-29T22:13:47 ) 2013-03-29T22:14:38 thestinger: yeah, thats true, but thats really not the same as the CPU guaranteeing a consistent operation of a memory location 2013-03-29T22:14:51 operation on* 2013-03-29T22:16:11 mcstar: http://en.cppreference.com/w/cpp/atomic/memory_order there's that crap ;p but it's much lower level and it can't be composed into something high-level 2013-03-29T22:16:24 * thestinger prefers to not think about that horrific stuff 2013-03-29T22:16:50 communication protocols are so much saner 2013-03-29T22:17:02 copy-free message passing 2013-03-29T22:17:28 i know a language that has that 2013-03-29T22:17:35 you know, it moves stuff even 2013-03-29T22:17:44 ;p 2013-03-29T22:17:48 iron oxidant 2013-03-29T22:19:06 Go can pass stuff without a copy too, because it fails at enforcing safety (it lets you pass a reference to mutable memory) 2013-03-29T22:19:18 but at that point you might as well be sharing memory... 2013-03-29T22:28:37 gn 2013-03-29T22:28:39 *** mcstar has quit IRC (Quit: mcstar) 2013-03-29T22:46:13 *** GarfTop has joined #aichallenge 2013-03-29T22:55:58 *** Garf has quit IRC (*.net *.split) 2013-03-29T23:45:49 *** smiley1983 has joined #aichallenge