CameraController, 3GS and 3.0

[natios]

I tried to do it from a timer called function.

- (void)frame
{
	CoreSurfaceBufferRef coreSurfaceBuffer = [cameraController performSelector:@selector(_createPreviewIOSurface)];
	uii = [cameraController performSelector:@selector(_createPreviewImage)];
	
	if (!coreSurfaceBuffer) 
		return;
	
		Surface *surface = [[Surface alloc]initWithCoreSurfaceBuffer:coreSurfaceBuffer];
		[surface lock];
		unsigned int height = surface.height;
		unsigned int width = surface.width;
		unsigned int alignmentedBytesPerRow = (width * 4);
		if (!readblePixels) {
			readblePixels = CGBitmapAllocateData(alignmentedBytesPerRow * height);
			NSLog(@"alloced readablepixels");
		}
		unsigned int bytesPerRow = surface.bytesPerRow;
		void *pixels = surface.baseAddress;
		for (unsigned int j = 0; j < height; j++) {
			memcpy(readblePixels + alignmentedBytesPerRow * j, pixels + bytesPerRow * j, bytesPerRow);
		}
		previewWidth = width;
		previewHeight = height;
		pixelDataLength = alignmentedBytesPerRow * height;
		previewBytesPerRow = alignmentedBytesPerRow;
		[surface unlock];
		[surface release];
}	

- (void)applicationDidFinishLaunching:(UIApplication *)application {    
	cameraController = [(id)objc_getClass("PLCameraController") performSelector:@selector(sharedInstance)];
	[cameraController setDelegate:self];
	[cameraController setFocusDisabled:NO];
	[cameraController setCaptureAtFullResolution:YES];
	[cameraController setDontShowFocus:YES];
	[cameraController retain];
	
	UIView *previewView = [cameraController performSelector:@selector(previewView)];
	[previewView retain];

	[window addSubview:previewView];
	[cameraController performSelector:@selector(startPreview)];
	
	sleep(2);
	
    [window makeKeyAndVisible];
	
	self.frameTimer = [NSTimer scheduledTimerWithTimeInterval:1.0/30.0 target:self selector:@selector(frame) userInfo:nil repeats:YES];
}

I then tried to print the data in readblePixels and pixels in various ways, all giving the pixels of a black image with a white alpha channel.
Could you please post your function using the _createPreviewIOSurface or tell me where I'm going wrong?

Thanks.

[rilocr]

Mmm... that should work...

When I was testing with that code, I used readblePixels for creating an UIImage and I was able to see the screen shot...

The only difference with my code is this line:

CoreSurfaceBufferRef *coreSurfaceBuffer = [cameraController _createPreviewIOSurface];

Try using a pointer.

[natios]

Thanks a lot rilocr.
I'm now able to get the data.
However, after about ten seconds the application is running, it crashes on _createPreviewIOSurface and gdb says:

Program received signal: “0”.
warning: check_safe_call: could not restore current frame

Got any idea what might mess up the stack?

[NitroPye]

Hello all (new to these parts),

I've successfully gotten access to the PLCameraController and am stuck where a lot of you are also stuck, trying to access raw image data.

The screen shot accessors from the PLCameraController object are unfortunetly useless to me as I need to modify the current frame, and display it. Unless I did some form of switching back and forth every frame to get a new screenshot but I am certain this would flicker like crazy.

I've tried taking the previewView UIView from the PLCameraController and rendering that but have had no luck. Here is the code I used in that attempt:

	UIGraphicsBeginImageContext(cameraView.bounds.size);
	[cameraView.layer renderInContext:UIGraphicsGetCurrentContext()];
	UIImage *contextImage = UIGraphicsGetImageFromCurrentImageContext();
	UIGraphicsEndImageContext();
	[imageView setImage:contextImage];

Unfortunetly all this does is return a completely black image. I am not even sure if this is a valid path to take for finding a fix.

Anyone have any thoughts on possibly getting camera frames from rendering previewView as a UIImage? Possible? Crazy? Wild goose hunt?

[rilocr]

Hi natios,

Try releasing the coreSurfaceBuffer variable with CFRelease(coreSurfaceBuffer).

That's the only additional code I have.

[iPhoneSDKDeveloper]

its hard to add on the extra bits of code you guys tell me. can one of you, who has successfully completed the plcameracontroller, send a download link with the xcode file so i can recieve it on a file sharing website. its really hard to code the camera controller.

[natios]

Thanks for your time rilocr.
You've helped me a lot.

[basm]

Hi troffmo5

Yes, _createPreviewImage is a screen capture, so if you have a navigation bar or a tool bar, they will appear on the image, I have tried also with _createPreviewIOSurface and it is the same, it's a screen capture.

Also, both functions return, what it seems, a preprocessed image/buffer, so the size is way too big than the size of the raw data buffer...

But, yes I think that's a solution, but at least, for my app, I really need the raw data :-(

Thanks

Hi there,

has anyone ever figured out a way to access the camera directly or find a workaround?
It is really annoying that you get a screen-capture when calling _createPreviewImage !! Whenever you overlay the cameras UIView with controls or opengl the image returned from _createPreviewImage is useless!

I tried to do a screen capture and only capture the camera UIView, but i can't use the cams UIView with:

UIGraphicsBeginImageContext(cam.bounds.size);
	[cam.layer renderInContext:UIGraphicsGetCurrentContext()];
	UIImage *resultingImage = UIGraphicsGetImageFromCurrentImageContext();
	UIGraphicsEndImageContext();

because it's not a full UIView and i don't know how to extend it to be able to use it with the screen capture mechanism.

Did any of you figure out how to get the cameras raw data or does anyone know how to extend the camdelegate to be able to take screenshots?

Greets
bas

[max.reif]

Hi,
i have an iphone 3g and want to make a video camera.
I can see the preview and i can get every 1/15 sec an UIImage ([cameraController _createPreviewImage]).
Now i want to save them because if they are stored in an array the iphone's memory is in less than 5 sec full.

Thans in advance
Max

Sorry, my English is so bad;
I'm german and 14 years old.

[rilocr]

Hi,

You can use ffmpeg, just google it. That would be an external library for your app, but I think there are other solutions.

Regards

[OniDaito]

I seem to be hitting this wall as well. trying to do AR apps using screengrabs is obviously impossible. I can see that the _camera variable was removed in FW3 which is proving to be a stumbling block. Has anybody managed to find the secret to grabbing the raw data yet and installing the needed callbacks?

Actually, I've noticed that Cycorder still works fine with my iPhone with FW3. I take it then, that Cycorder uses a different method than the one we are talking about here? If so, does anyone know the method used there? Personally, I don't care how it's done, so long as I can get to play with some AR.

[rilocr]

I read in another forum the Cycorder does access the camera directly and also it uses the hardware MPEG encoder provided by the Infineon PMB8876 S-GOLD 2 chip onboard, but it seems Saurik doesn't want to share his secret, obviously...

[OniDaito]

Yeah, he aint talkin' about that sort of thing. I suspect it's some evil, low-level c hacking with the Jailbreak stuff. It's certainly an option I'm considering anyway. Theres probably a /dev style device he's playing with. Regardless I'd like to use the non-jailbreak method but the removal of _camera is proving annoying. Damn you Apple!

[kraft2000]

Hey,

I'm using the same kind of code you used (with a _createPreviewIOSurface inside a timer), and I just found a way to superpose drawings without affecting the Preview image.

You just have to add a UIWindow, with a transparent view, and draw inside that view :

	cameraController = [(id)objc_getClass("PLCameraController") performSelector:@selector(sharedInstance)];
	[cameraController retain];
	[cameraController setDelegate:self];
	[cameraController setFocusDisabled:NO];
	[cameraController setDontShowFocus:YES];
	[cameraController setCameraMode:1];
	
	previewView = [self.cameraController performSelector:@selector(previewView)];
	[previewView retain];
	previewView.center = CGPointMake(previewView.center.x, previewView.center.y + 43);
	[self.view addSubview:previewView];
	
	myImageView = [[MyImageView alloc] initWithFrame:previewView.frame];
	myImageView.center = CGPointMake(myImageView.center.x, myImageView.center.y + 20);
	myImageView.backgroundColor = [UIColor clearColor];

	myWindow = [[UIWindow alloc] init];
	[myWindow addSubview:myImageView];

	[myWindows makeKeyAndVisible];

with MyImageView a UIView class in which I derived the drawRect method, and with a call to [faceImageView setNeedsDisplay] inside the timer method.

It's still not the best method ever to grab the camera buffer, but at least I can start displaying things without affecting my algorithm.

[OniDaito]

Have I understood you rightly? Does this method allow you to sample the CameraView without sampling the stuff you draw over the top?

[kraft2000]

yep, just an a new UIwindow and draw stuff you want inside it.

[OniDaito]

Cant remember if I ever got the PLCameraController to work. I presume this is using iPhone SDK/OS 2?

Ok - I think it've got it:

Basically, I'm bypassing the header file entirely – instead instantiating the camera controller using objc_getClass.

My init code is thus:

self.cameraController = [(id)objc_getClass("PLCameraController") performSelector:@selector(sharedInstance)];
		[cameraController setDelegate:self];
		[cameraController setFocusDisabled:NO];
		[cameraController setCaptureAtFullResolution:YES];
		[cameraController setDontShowFocus:YES];
		
		UIView *previewView = [cameraController performSelector:@selector(previewView)];
		[containerView addSubview:previewView];
		[cameraController performSelector:@selector(startPreview)];
		sleep(2);

I tried this but my self.cameraController is always null. I suspect there is something I'm missing; a framework maybe?

Sorry guys, just figured it out. PhotoLibrary framework for OS 3.1.2 fixes this to a degree.... I end up with a black screen with the classic white bar at the bottom. Argh!

[rilocr]

Hi!

Have any of you seen the app Ustream Live Broadcaster from the App Store?

They make video live streaming!! And the performance is really good!

That means Apple now allows apps that make live streaming!

[OniDaito]

It appears to be taking a sample of the camera, most likely with takePicture() calls. I can't see how fast the framerate is but its not brilliant (don't get me wrong though, I'm really impressed with the work). But, I suspect the overlays are being cropped out of the final frames sent to the server.

[rilocr]

Mmm... but I think takePicture() is not as fast as the streammed video, I will make some tests...

Any other way to take a sample of the camera, without using a private API?